Fixtureless lensmeter system

ABSTRACT

A lensmeter system may include a mobile device having a camera. The camera may capture a first image of a pattern through a lens that is separate from the camera, while the lens is in contact with a pattern. The mobile device may determine the size of the lens based on the first image and known features of the pattern. The camera may capture a second image of the pattern, while the lens is at an intermediate location between the camera and the pattern. The second image may be transformed to an ideal coordinate system, and processed determine a distortion of the pattern attributable to the lens. The mobile device may measure characteristics of the lens based on the distortion. Characteristics of the lens may include a spherical power, a cylinder power, and/or an astigmatism angle.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. patent application Ser. No.16/164,488, filed Oct. 18, 2018, which is a Continuation-in-Part of U.S.patent application Ser. No. 16/022,324, filed on Jun. 28, 2018, which isa Continuation of U.S. patent application Ser. No. 15/157,835 filed onMay 18, 2016 and issued as U.S. Pat. No. 10,036,685 on Jul. 31, 2018,each of which is hereby incorporated by reference herein in itsentirety.

BACKGROUND Technical Field

The technical field generally relates to determining prescriptions ofcorrective lenses, and more particularly, in one aspect, to mobiledevice lensmeters and methods of operating such lensmeters.

Background Discussion

Eye doctors, eyeglass lens makers, and others who work with lenses oftenuse traditional lensmeters to determine the prescription (including thespherical power, cylinder power, and axis) of an unknown correctivelens. Such lensmeters typically involve shining a light source through apattern and a corrective lens mounted on a fixture of the lensmeter, andviewing the light at an eyepiece opposite the light source. Observingthe pattern's distorted appearance through the eyepiece, the distortioncan be correlated to a prescription known to create such a distortion.

A fixture holds the pattern, the corrective lens, and the eyepiece in anappropriate spacing and configuration to one another. Yet the fixture istypically large and heavy, making such an arrangement unwieldy andundesirable for use at home or in the field. Such traditional methods ofdetermining a prescription for a corrective lens also do not provide aconvenient way to convey the prescription information to others, such asan eye doctor or lens maker. While the information may be conveyed bytelephone, for example, the risk of transcription error or other issuesrises, making it less attractive for individuals to determine acorrective lens prescription in a convenient setting, such as home orwork. Those seeking to determine a prescription of an unknown correctivelens must therefore travel to an eye doctor or other professional, whichintroduces additional delays and costs to the process.

SUMMARY

According to one aspect, a process for determining characteristics of alens includes obtaining a captured image of a pattern through acorrective lens; transforming the captured image to an ideal coordinatesystem; processing the captured image to determine an overall distortionfrom a reference pattern to the pattern of the captured image;determining a distortion of the captured pattern attributable to thecorrective lens; and measuring at least one characteristic of thecorrective lens. According to one embodiment, the captured imageincludes a first region containing the pattern and created by lightpassing through the corrective lens, and a second region created bylight not passing through the corrective lens, and determining thedistortion of the captured pattern attributable to the corrective lensis performed at least in part with reference to the second region.According to a further embodiment, the pattern is a checkerboardpattern, and the second region contains a border. According to anotherembodiment, transforming the captured image to an ideal coordinatesystem includes detecting a plurality of captured reference landmarks inthe second region of the captured image; determining a transformationfrom a plurality of ideal reference landmarks to the plurality ofcaptured reference landmarks; and applying the transformation to thecaptured image.

According to another embodiment, the pattern is a first pattern and thecorrective lens is a first corrective lens, and obtaining the capturedimage of the pattern through the corrective lens includes obtaining acaptured image of the first pattern through the first corrective lensand a second pattern through the second lens.

According to yet another embodiment, processing the captured image todetermine the overall distortion from the reference pattern to thepattern of the captured image includes detecting, in the captured image,a plurality of captured pattern landmarks; determining a transformationfrom a plurality of ideal pattern landmarks to the plurality of capturedpattern landmarks; and determining for the corrective lens, from thetransformation, a spherical power measurement, a cylinder powermeasurement, and an astigmatism angle measurement. According to afurther embodiment, the transformation is a dioptric power matrix.

According to yet a further embodiment, obtaining the captured image ofthe at least one pattern through the corrective lens is performed at afirst location of a camera lens relative to the at least one pattern,further including capturing, at a second location of the camera lensrelative to the at least one pattern, a second captured image of the atleast one pattern through the corrective lens; detecting, in the secondcaptured image, the plurality of captured pattern landmarks; determininga second transformation from the plurality of ideal pattern landmarks tothe plurality of captured pattern landmarks; determining, for thecorrective lens, from the second transformation, the spherical powermeasurement, the cylinder power measurement, and the astigmatism anglemeasurement; and selecting a preferred transformation from the firsttransformation and the second transformation for which the sphericalpower measurement and the cylinder power measurement have an extremevalue.

According to a still further embodiment, the captured image is capturedby a camera having a camera lens, and the corrective lens is positionedat a known location relative to the camera lens and the pattern.According to a further embodiment, determining the distortion of thecaptured image attributable to the corrective lens includes determininga distance between the camera lens and the pattern; and determining atleast one focal length of the corrective lens with reference to thedistance, the spherical power measurement, and the cylinder powermeasurement.

According to one embodiment, measuring the at least one characteristicof the corrective lens includes determining a prescription of thecorrective lens, the prescription including at least a sphere value, acylinder value, and an axis value. According to another embodiment,obtaining a captured image of a pattern through a corrective lensincludes obtaining, through a camera lens, a captured image of a firstpattern through a first corrective lens and a second pattern through asecond corrective lens, wherein the two patterns are spaced from eachother such that obtaining the captured image of the first patternthrough the first corrective lens and the second pattern through thesecond corrective lens can be performed when the first corrective lensand the second corrective lens are positioned at a known locationrelative to the camera lens and the first and second patterns.

According to yet another embodiment, the process further includesdetermining, from the captured image, a first location of a camera lensof a lensmeter through which the captured image was captured;identifying a direction to a second location relative to the firstlocation; guiding a user of the lensmeter to the second location; andcapturing a second captured image of the pattern through the correctivelens.

According to another aspect, a lensmeter includes a camera; a visualdisplay; and a processor coupled to the camera and configured to obtaina captured image of a pattern through a corrective lens; transform thecaptured image to an ideal coordinate system; process the captured imageto determine an overall distortion from a reference pattern to thepattern of the captured image; determine a distortion of the capturedpattern attributable to the corrective lens; and measure at least onecharacteristic of the corrective lens.

According to one embodiment, the captured image includes a first regioncontaining the pattern and created by light passing through thecorrective lens, and a second region created by light not passingthrough the corrective lens. According to a further embodiment, theprocessor is further configured to transform the captured image to anideal coordinate system by being configured to detect a plurality ofcaptured reference landmarks in the second region of the captured image;determine a transformation from a plurality of ideal reference landmarksto the plurality of captured reference landmarks; and apply thetransformation to the captured image.

According to another embodiment, the processor is further configured toprocess the captured image to determine the overall distortion from thereference pattern to the pattern of the captured image by beingconfigured to detect, in the captured image, a plurality of capturedpattern landmarks; determine a transformation from a plurality of idealpattern landmarks to the plurality of captured pattern landmarks; anddetermine for the corrective lens, from the transformation, a sphericalpower measurement, a cylinder power measurement, and an astigmatismangle measurement. According to a further embodiment, the processor isfurther configured to obtain the captured image of the at least onepattern through the corrective lens at a first location, the processorfurther configured to capture, at a second location, a second capturedimage of the at least one pattern through the corrective lens; detect,in the second captured image, the plurality of captured patternlandmarks; determine a second transformation from the plurality of idealpattern landmarks to the plurality of captured pattern landmarks;determine, for the corrective lens, from the second transformation, thespherical power measurement, the cylinder power measurement, and theastigmatism angle measurement; and select a preferred transformationfrom the first transformation and the second transformation for whichthe spherical power measurement and the cylinder power measurement havean extreme value. According to yet a further embodiment, the capturedimage is captured through a camera lens of the camera, and the processoris further configured to determine the distortion of the captured imageattributable to the corrective lens by being configured to determine adistance between the camera lens and the pattern; and determine at leastone focal length of the corrective lens with reference to the distance,the spherical power measurement, and the cylinder power measurement.

According to one embodiment, the processor is further configured tomeasure the at least one characteristic of the corrective lens by beingconfigured to determine a prescription of the corrective lens, theprescription including at least a sphere value, a cylinder value, and anaxis value. According to another embodiment, the pattern is printed on aphysical medium. According to yet another embodiment, the pattern isdisplayed on an electronic display device.

According to some aspects, a method of operating a lensmeter system isprovided, the method including capturing, with a camera of the lensmetersystem, a first image of a pattern through a corrective lens that is incontact with the pattern. The method also includes determining, with acomputing device of the lensmeter system, a size of the corrective lensbased on the first image and the pattern. The method also includescapturing, with the camera, a second image of the pattern through thecorrective lens while the corrective lens is at an intermediate positionbetween the camera and the pattern. The method also includesdetermining, with the computing device, a distortion, attributable tothe corrective lens, of the pattern in the second image, using thedetermined size of the corrective lens. The method also includesmeasuring, with the computing device, at least one characteristic of thecorrective lens based on the determined distortion.

According to other aspects, a mobile device is provided that includes acamera and a processor configured to obtain, from the camera, a firstimage of a pattern through a corrective lens that is in contact with thepattern. The processor is further configured to determine a size of thecorrective lens based on the first image and the pattern. The processoris further configured to obtain, from the camera, a second image of thepattern through the corrective lens with the corrective lens at anintermediate position between the camera and the pattern. The processoris further configured to determine a distortion, attributable to thecorrective lens, of the pattern in the second image, using thedetermined size of the corrective lens. The processor is furtherconfigured to measure at least one characteristic of the corrective lensbased on the determined distortion.

According to other aspects a lensmeter system is provided that includesa pattern having features with sizes, and a mobile device. The mobiledevice includes a camera, memory storing information associated with thepattern and the sizes of the features, and a processor configured tocapture, using the camera, a first image of a pattern through acorrective lens that is in contact with the pattern. The processor isfurther configured to determine a size of the corrective lens based onthe first image and the pattern using the information associated withthe pattern and the sizes of the features. The processor is furtherconfigured to capture, using the camera, a second image of the patternthrough the corrective lens while the corrective lens is at anintermediate position between the camera and the pattern. The processoris further configured to determine a distortion, attributable to thecorrective lens, of the pattern in the second image, using thedetermined size of the corrective lens. The processor is furtherconfigured to measure at least one characteristic of the corrective lensbased on the determined distortion.

Still other aspects, embodiments, and advantages of these exemplaryaspects and embodiments, are discussed in detail below. Moreover, it isto be understood that both the foregoing information and the followingdetailed description are merely illustrative examples of various aspectsand embodiments, and are intended to provide an overview or frameworkfor understanding the nature and character of the claimed subjectmatter. Particular references to examples and embodiments, such as “anembodiment,” “an example,” “one example,” “another embodiment,” “anotherexample,” “some embodiments,” “some examples,” “other embodiments,” “analternate embodiment,” “various embodiments,” “one embodiment,” “atleast one embodiments,” “this and other embodiments” or the like, arenot necessarily mutually exclusive and are intended to indicate that aparticular feature, structure, or characteristic described in connectionwith the embodiment or example and may be included in that embodiment orexample and other embodiments or examples. The appearances of such termsherein are not necessarily all referring to the same embodiment orexample.

Furthermore, in the event of inconsistent usages of terms between thisdocument and documents incorporated herein by reference, the term usagein the incorporated references is supplementary to that of thisdocument; for irreconcilable inconsistencies, the term usage in thisdocument controls. In addition, the accompanying drawings are includedto provide illustration and a further understanding of the variousaspects and embodiments, and are incorporated in and constitute a partof this specification. The drawings, together with the remainder of thespecification, serve to explain principles and operations of thedescribed and claimed aspects and embodiments.

BRIEF DESCRIPTION OF DRAWINGS

Embodiments of the invention are not limited to the details ofconstruction and the arrangement of components set forth in thefollowing description or illustrated in the drawings. Embodiments of theinvention are capable of being practiced or of being carried out invarious ways. Also, the phraseology and terminology used herein is forthe purpose of description and should not be regarded as limiting. Theuse of “including,” “comprising,” or “having,” “containing,”“involving,” and variations thereof herein, is meant to encompass theitems listed thereafter and equivalents thereof as well as additionalitems.

Various aspects of at least one embodiment are discussed below withreference to the accompanying figures, which are not intended to bedrawn to scale. The figures are included to provide an illustration anda further understanding of the various aspects and embodiments, and areincorporated in and constitute a part of this specification, but are notintended as a definition of the limits of any particular embodiment. Thedrawings, together with the remainder of the specification, serve toexplain principles and operations of the described and claimed aspectsand embodiments. In the figures, each identical or nearly identicalcomponent that is illustrated in various figures is represented by alike numeral. For purposes of clarity, not every component may belabeled in every figure. In the figures:

FIG. 1 is an illustration of a prior art lensmeter;

FIG. 2 is a block diagram of a lensmeter system according to one or moreembodiments;

FIG. 3 is a block diagram of a mobile device lensmeter according to oneor more embodiments;

FIG. 4 is a flow chart of a method for operating a mobile devicelensmeter according to one or more embodiments;

FIG. 5A is an illustration of a reference pattern group according to oneor more embodiments;

FIG. 5B is an illustration of a captured image of the reference patterngroup of FIG. 5A according to one or more embodiments;

FIG. 5C is the captured image of FIG. 5B after transformation to anideal coordinate system; and

FIG. 6 illustrates a number of pattern landmarks for a reference patterngroup and a pattern of a captured image according to one or moreembodiments.

FIG. 7 illustrates a perspective view of a fixtureless lensmeter systemin a first configuration in accordance with various aspects of thedisclosure.

FIG. 8 illustrates top perspective view of a fixtureless lensmetersystem in a second configuration in accordance with various aspects ofthe disclosure.

FIG. 9 illustrates a mobile device user-interface view during afixtureless lensmeter operation in accordance with various aspects ofthe disclosure.

FIG. 10 is a flow chart of illustrative operations that may be performedfor a fixtureless lensmeter operation in accordance with various aspectsof the disclosure.

FIG. 11 illustrates an example architecture for a fixtureless lensmetersystem suitable for practicing some implementations of the disclosure.

DETAILED DESCRIPTION

According to one or more embodiments, the processes and systemsdisclosed allow a lensmeter device such as a mobile phone to determinecharacteristics, such as a prescription, of one or more correctivelenses. In some embodiments, an image of one or more patterns iscaptured through the corrective lens by a camera device, and thedistortion of the patterns is measured to determine the characteristicsof the corrective lens by a connected computing device with specializedsoftware. Embodiments discussed herein describe a lensmeter as a deviceconfigured to measure characteristics of one or more corrective lenseswithout requiring the specific spacing and arrangement required by knownlensmeters and enforced by the fixtures they incorporate. The presentlensmeter may be a smartphone or tablet device on which specializedsoftware (e.g., an app) is installed for performing the claimed methods.In some operational scenarios, all of the processing for determining thecharacteristics of the corrective lens are performed at the smartphoneor the tablet (e.g., by the processor(s) of the smartphone or thetablet). In other operational scenarios, data such as camera images orimage metadata may be communicated between the smartphone or the tabletand one or more remote processors such as one or more cloud servers thatperform some or all of the processing for determining thecharacteristics of the corrective lens. In these other operationalscenarios, information indicating the determined characteristics can becommunicated from the remote processor(s) to the smartphone, the tablet,and/or other devices or systems. In some implementations, the lensmetermay include one or more components that have a fixed location (e.g., acamera embedded in a wall or fixture and communicatively coupled to oneor more local and/or remote processors) and that can measurecharacteristics of corrective lenses without requiring the correctivelens and the pattern to be precisely spaced and arranged relative to thelensmeter. Such an arrangement may be suitable, for example, in a retailenvironment, such as an optician location or eyeglass retailer.

The patterns may be displayed on a piece of paper, or may be displayedon a display of another device, such as a laptop computer. In someembodiments, the mobile device (i.e., the mobile lensmeter) and otherdevices (e.g., the other device displaying the pattern) may be paired,to allow the devices to communicate and interact during the measurementprocess. Examples herein depicting the mobile lensmeter as the mobiledevice itself are for illustrative purposes only, and it will beappreciated that functionality discussed herein with respect to the“mobile lensmeter” may be performed on, or in conjunction with, suchother devices as part of a mobile lensmeter system.

In some embodiments, two patterns are spaced and configured such thatthey are visible to the mobile lensmeter—each through one of a pair ofcorrective lenses in an eyeglass frame—when the corrective lenses arepositioned approximately halfway between the patterns and the lensmeterand oriented appropriately. Such an arrangement allows for easy,intuitive positioning of the mobile lensmeter, the patterns, and thecorrective lenses. Furthermore, the mobile lensmeter is configured todetermine the distance to the pattern and take that measurement intoaccount when determining the prescription. This design facilitates themanual positioning of the elements, eliminating the need for a fixture.In one embodiment, the pattern is a rectangle displayed on a physicalmedium or on a computer display. In some embodiments, the pattern issurrounded by a border having reference landmarks or other features usedto orient the captured image.

According to one or more embodiments, the disclosed processes andsystems transform the captured image to an ideal coordinate system tocompensate for the orientation of the lensmeter relative to the patternduring the image capture process. In some embodiments, thetransformation is made with reference to the location of referencelandmarks in the captured image relative to the location of referencelandmarks in a reference pattern group.

According to one or more embodiments, the disclosed processes andsystems process the captured image to determine an overall distortion bydetecting and determining the location of a number of captured patternlandmarks in the captured image. The system determines a transformationthat describes the distortion from the location of a number of referencepattern landmarks (in the ideal coordinate system) relative to thecorresponding captured pattern landmarks in the captured image. Anexpression of the transformation (e.g., a dioptric power matrix) may beused to determine measurements of the corrective lens, including aspherical power, a cylinder power, and an astigmatism angle. The portionof the overall distortion due to the corrective lens (as opposed to thelens of the lensmeter) may be determined in part by determining at leastone focal length of the corrective lens. Other characteristics of thecorrective lens may also be measured. The present embodiments are notlimited to sphero-cylindrical lenses, and may be suitable for lenseshaving other characteristics, such as single vision lenses, bifocallenses, trifocal lenses, progressive lenses, adjustable focus lenses, orlenses that correct for higher order aberrations.

According to one or more embodiments, multiple images may be capturedand analyzed to identify an image captured in a preferred positionand/or orientation, e.g., where the corrective lens is closest tohalfway between the lensmeter and the pattern.

According to one or more embodiments, a lensmeter is provided thatincludes a camera, a visual display, and a processor configured to carryout the processes described herein. The lensmeter may be a dedicatedlensmeter, or may be mobile device (e.g., a smartphone or a tabletdevice) executing lensmeter software, such as a downloadable app.

FIG. 1 illustrates a conventional optical lensmeter system 100 fordetermining a prescription and/or other unknown characteristics of acorrective lens 130. A light source 110 is directed through a pattern120 (e.g., a transparent target having a printed pattern with knowndimensions and arrangement) and the corrective lens 130 (and a number ofstandard and objective lenses, not shown) to an eyepiece 140. A viewerwhose eye is engaged with the eyepiece 140 can observe the way that thecorrective lens 130 distorts the light passing through the pattern 120.By measuring the distortive effect of the corrective lens 130, the usercan determine certain characteristics of the corrective lens 130,including the spherical power, cylindrical power, and axis measurementsof the corrective lens 130. The lensmeter system 100 requires a fixture150, including a lens holder 152, to maintain the pattern 120, thecorrective lens 130, and the eyepiece 140 in a precisely spaced andoriented arrangement. The optical principles underlying the operation ofthe lensmeter system 100 require that the specific spacing andorientation be maintained by the fixture 150.

Similarly, digital lensmeters can be used to image a pattern through asingle corrective lens, and use the distortion in the image to determinea prescription and/or other unknown characteristics of a correctivelens. Like conventional optical lensmeters, currently available digitallensmeters require a fixture for holding the corrective lens, thelensmeter lens, and the pattern in a precisely spaced and orientedarrangement.

FIG. 2 illustrates a block diagram of a lensmeter system 200 accordingto one or more embodiments. In the embodiments shown in FIG. 2, thesystem 200 includes a lensmeter 210, a corrective lens 220, and apattern 230. In operation, the lensmeter 210 captures an image of thepattern 230 through the corrective lens 220. The corrective lens 220distorts the light emitted by, or reflecting off the pattern 230 intothe lensmeter 210, and the distortive effect may be measured in order todetermine one or more unknown characteristics of the corrective lens220, including the sphere, cylinder, and axis measurements.

The captured image of the pattern 230 is normalized by converting it toan ideal coordinate system using reference landmarks near the pattern230. The normalization compensates for rotation, tilt, or distancevariances in the spacing and orientation among the lensmeter 210, thecorrective lens 220, and the pattern 230. No fixture is thereforerequired in the lensmeter system 200. The normalized pattern 230 canthen be compared to a reference pattern, also in the ideal coordinatesystem, and the distortive effect of the corrective lens can be isolatedfrom the distortive effect of the lens of the lensmeter 210 itself.

In some embodiments, the pattern 230 is displayed on an electronicdisplay (not shown), such as a computer monitor, tablet or other mobiledevice, or the like, or is projected onto a surface by a projector. Forexample, the pattern 230 may be provided on a website accessible by thelensmeter system 200, or may be provided by or through a mobile apprunning on a mobile device. In other embodiments, the pattern 230 isprinted on a physical medium such as a piece of paper or plastic.

In some embodiments, two or more patterns may be used to allow fordetermining the characteristics of two or more corrective lensessimultaneously. In a preferred embodiment, two spaced patterns are used,with each pattern 230 being a rotation-variant checkerboard grid ofalternating black and white squares, where the number of rows in thecheckerboard differs from the number of columns by 1. Thisrotation-variant quality allows the lensmeter 210 to determine whetherthe pattern 230 is being viewed in a correct, upright position, oralternately is rotated on its side. In one embodiment, the pattern 230is a black-and-white checkerboard design having eight (8) rows and seven(7) columns. In another embodiment, the pattern 230 has 16 rows and 15columns. Other configurations or color combinations are also possible.

FIG. 3 is a block diagram of a lensmeter 210 according to someembodiments. In some embodiments, the lensmeter 210 is a consumer mobiledevice (e.g., a smartphone or tablet device) or computer (e.g., a laptopcomputer) running specialized software to perform the operationsdescribed herein. In other embodiments, the lensmeter 210 is a dedicatedlensmeter device. The lensmeter 210 includes a camera 310 having a lens312, and further includes a processor 320, a user interface 330, anetwork interface 340, a memory 350, and lensmeter software 360. In someembodiments, the camera 310 is an integral component of the lensmeter210. In other embodiments, the camera 310 may be an add-on component oraccessory. The processor 320 is coupled to the camera 310 and executessoftware 360 for the image capturing functions performed by thelensmeter 210. In some embodiments, the functionality of the lensmeter210 may be performed in conjunction with other devices as part of abroader lensmeter system. For example, in an embodiment wherefunctionality of the lensmeter 210 is performed by a user's smartphone,the smartphone may be paired with the user's laptop computer in order tocontrol the display of the patterns. In that example, the lensmeter 210may be considered to include both the user's smartphone and the laptopcomputer.

The user interface 330 receives input from, and provides output to, auser of the lensmeter 210. In some embodiments, the user interface 330displays to the user the image currently visible through the lens 312 ofthe camera 310, allowing the user to adjust the position or orientationof the lensmeter 210. In some embodiments, the user interface 330provides the user with a physical or on-screen button to interact within order to capture an image. In other embodiments, the image iscaptured automatically when the pattern 230 is detected in the image andcertain alignment, size, lighting, and resolution conditions are met.

The user interface 330 may also provide indications to the user to moveany of the lensmeter 210, the corrective lens 220, and the pattern 230to a different absolute position or orientation, or to a differentposition or orientation relative to each other. For example, the userinterface 330 may provide directions such as “MOVE FORWARD,” “MOVEBACKWARD,” “TILT LENSMETER FORWARD,” instructions conveyed with graphicsand illustrations, or other such directions, until the user haspositioned the lensmeter 210 such that the corrective lens 220 ispositioned at an optimal, known position relative to the lensmeter 210and the pattern 230; and until the lensmeter 210, the corrective lens220, and the pattern 230 are aligned so that the pattern 230 is viewablethrough the corrective lens 220 at the lensmeter 210. In someembodiments, the user interface 330 and/or other components of thelensmeter 210 may provide such instructions audibly, such as by recordedvoice instructions, or by an audible tone emitted at a frequencyproportional (or inversely proportional) to the distance of thelensmeter 210 from the correct position. In other embodiments, the userinterface 330 may provide an indication to the user once the user hascorrectly positioned the lensmeter 210, the corrective lens 220, and thepattern 230, for example by displaying a “green light” or thumbs-upicon. The user interface 330 may also allow a user to interact withother systems or components, such as by giving an instruction totransmit corrective lens prescription information to the user's doctor.

In some embodiments, the network interface 340 allows for access todownloads and upgrades of the software 360. In some embodiments, one ormore steps of the process described below may be performed on a server(not shown) or other component distinct from the lensmeter 210, and datamay be passed between the lensmeter 210 and the server via the networkinterface 340. The network interface 340 may further allow forautomatically uploading lens characteristics or prescription informationto another entity, e.g., the user's optometrist or another correctivelens provider.

Some or all of the processes described herein, as well as otherfunctions, may be performed by the lensmeter software 360 executing onthe lensmeter 210, or by other systems in communication with thelensmeter 210 (e.g., via the network interface 340).

FIG. 4 is a flow chart of a process 400 for determining characteristicsof a corrective lens according to one or more embodiments. Suchembodiments may be implemented using a system such as that shown inFIGS. 2 and 3.

The process begins at step 410.

At step 420, a captured image of a pattern is obtained through acorrective lens. The image is captured by a camera (e.g., camera 310).In some embodiments, the camera is part of, or attached to, a dedicatedlensmeter device. In other embodiments, the camera is part of a mobiledevice (e.g., a smartphone or tablet device). In some embodiments, theuser is instructed to hold the mobile device with the camera orientedtoward the pattern such that the pattern is viewed through thecorrective lens. An image of the pattern is then captured by the camera.The image may be captured in response to a user indication, such asclicking a physical button or an interface element on the screen of themobile device. In other embodiments, the image may be capturedautomatically once a stable, relatively static image has been obtainedand is in focus, and the lensmeter, corrective lenses, and pattern areappropriately aligned. For example, an accelerometer of the mobiledevice may be used to determine that the camera is relatively still. Ifa focused image can be obtained, the system may attempt to discern thepresence of the pattern within the image using known image processingand detection techniques. In some embodiments, multiple images may becaptured, and an image may be selected for further processing from amongthe multiple images based on such criteria as the image in which thepattern is most in focus, whether the elements are appropriatelyaligned, or the like.

In some embodiments, the object may be an image displayed on a computerdisplay. The object may be a pattern with a known geometry and easilydetectable feature points. According to some embodiments a checkerboardpattern is used.

FIG. 5A illustrates a pattern group 500 in which patterns 510, 520 arepositioned to be used in simultaneously detecting characteristics of twocorrective lenses, such as two eyeglass lenses within an eyeglass frame.The patterns 510, 520 are positioned within a border 530. The border 530includes border reference landmarks 532, 533, 534, 535 at knownlocations relative to the border 530. The border 530 and/or the borderreference landmarks 532, 533, 534, 535 are used to correct theorientation of the captured image in subsequent steps. In a preferredembodiment, four border reference landmarks are used, though someembodiments may use as few as two border reference landmarks. In oneembodiment, a border reference landmark 532, 533, 534, 535 is located ineach of the four interior corners of the border 530. The borderreference landmarks 532, 533, 534, 535 may be markers recognizable inthe captured image using computer vision techniques, or may be inherentlandmarks detected by computer vision techniques and/or with referenceto the known geometry of the pattern group 500. For example, if it isknown that the border 530 is a rectangle having four interior corners,those four interior corners may be located and used as the borderreference landmarks 532, 533, 534, 535.

The patterns 510, 520 also include a plurality of pattern referencelandmarks 512. The locations of the plurality of pattern referencelandmarks 512 in the captured image are used in subsequent steps todetermine the nature of the distortion introduced by the correctivelens. In some embodiments, a pattern reference landmark 512 is locatedat adjoining corners of squares within a checkerboard pattern. Thepattern reference landmarks 512 may be markers recognizable in thecaptured image using computer vision techniques, or may be landmarksdetected by computer vision techniques and/or with reference to theknown geometry of the pattern group 500.

The locations of the border reference landmarks 532, 533 and the patternreference landmarks 512 are known in the pattern group 500. Those knownlocations allow the pattern group 500 to be used as a reference patterngroup in subsequent steps, against which the locations of those samepoints in captured images may be compared.

In some embodiments, the lensmeter is configured to operate when thecorrective lenses in an eyeglass frame are halfway between the lensmeterand the patterns 510, 520. The patterns 510, 520 are configured andspaced such that each of the patterns 510, 520 is aligned with both thelensmeter and one of the corrective lenses when the corrective lensesare halfway between the lensmeter and the patterns 510, 520. Such anarrangement can be achieved, for example, when the distance between thecenters of the patterns 510, 520 is twice as large as the distancebetween the centers of the corrective lenses. For example, if thedistance between the centers of corrective lenses in an eyeglass frameis 77.5 mm, then the patterns 510, 520 may be spaced such that thedistance between the centers of the patterns 510, 520 is 77.5×2=155 mm.

The patterns 510, 520 and/or the border 530 are sized and configuredsuch that, when the lensmeter captures an image of the patterns throughthe corrective lenses, the openings in a normal-sized eyeglass framewholly surround the patterns in the captured image, meaning the patternsare completely overlaid by the corrective lenses. The openings of theeyeglass frame are in turn wholly contained within the border 530. Thecaptured image can be considered as having one or more first regionscreated from light passing through (i.e., distorted by) the correctivelenses, and one or more second regions created from light passing around(i.e., undistorted by) the corrective lenses.

A captured image 550 illustrating such a configuration can be seen inFIG. 5B. The pattern 510 is wholly contained within an opening 542 of aneyeglass frame 540, and the pattern 520 is wholly contained within anopening 544 of the eyeglass frame 540. The patterns 510, 520 in thecaptured image 550 have been distorted by the corrective lenses in theeyeglass frame 540. The eyeglass frame 540 is wholly contained withinthe border 530. By employing such a configuration, the distortion of thepatterns 510, 520 in the captured image due to the corrective lenses canbe measured, whereas the border reference landmarks 532, 533 remainundistorted.

It will be appreciated that the pattern group 500 illustrated in FIGS.5A and 5B is for illustrative purposes, and different configurations,sizes, or types of patterns and/or borders may be employed, or omittedaltogether, within the scope of the disclosure. In some embodiments,more than one image may be captured, with each image cropped orotherwise limited to contain only one pattern. In other embodiments, oneimage of both patterns is captured, and the image split into two imagesfor parallel processing on each pattern in subsequent steps. In stillother embodiments, video clips of the patterns may be captured, ormultiple static images captured in rapid succession.

It will also be appreciated that lenses having different characteristicswill distort the pattern in the captured image in different ways. Forexample, lenses with positive powers will magnify the pattern in thecaptured image, causing the pattern to appear larger through thecorrective lens. In that situation, the pattern may be sized such thatthe pattern in the captured image is not too large to be fully boundedby the corrective lens. Similarly, lenses with negative powers willdiminish the pattern in the captured image, causing the pattern toappear smaller through the corrective lens. In that situation, thepattern may be sized such that the pattern in the captured image is nottoo small to be identified and processed in identified in later steps.Accordingly, in some embodiments the pattern may be displayed on adisplay device allowing the pattern size to be configured according tothe characteristics of the lens or other considerations. In someembodiments, the user may be provided an interface (either via thedisplay device or the lensmeter 210) to resize the pattern, or to selecta characteristic of the lens and cause a suitably-sized pattern to bedisplayed. In other embodiments, the pattern may be resizedautomatically by the system so that it is the correct size in thecaptured image.

As can be seen in the captured image 550 in FIG. 5B, the border 530 isrotated counter-clockwise from the horizontal rectangular configurationshow in FIG. 5A, and the patterns 510, 520 and border 530 in FIG. 5B aresmaller than their counterparts in FIG. 5A. These variations make itdifficult to directly compare the pattern group 500 of the capturedimage 550 to the reference pattern group 500.

Therefore, returning to FIG. 4, at step 430, the captured image istransformed to an ideal coordinate system. In some embodiments, thecaptured image is transformed to the ideal coordinate system representedby the reference pattern group 500 of FIG. 5A. This transformation mayinvolve rotating, resizing, cropping, and skewing the image to removeany distortions or imprecisions introduced by the image capture process.In some embodiments, the captured image is transformed to an idealcoordinate system by detecting the border reference landmarks 532′,533′, 534′, 535′ in the captured image 550, and transforming thecaptured image 550 using image manipulation techniques to cause theborder reference landmarks 532′, 533′, 534′, 535′ to appear in the samelocation in the captured image as the corresponding border referencelandmark 532, 533, 534, 535 in the reference pattern group 500 of FIG.5A. The border reference landmarks 532′, 533′, 534′, 535′ may bedetected by computer vision techniques, and the border referencelandmarks 532′, 533′, 534′, 535′ or the pixels constituting them may beconfigured to have a shape, color, or other characteristic suitable forperforming such computer vision techniques.

In some embodiments, a matrix transform is determined from the distancebetween the border reference landmarks 532, 533, 534, 535 in thecaptured image 550 and the corresponding border reference landmarks 532,533, 534, 535 in the reference pattern group 500 of FIG. 5A. The matrixtransform is then applied to some or all of the pixels of the capturedimage 550 in order to effect the transformation.

The captured image 550 as it appears after transformation to the idealcoordinate system can be seen in FIG. 5C. The border reference landmarks532′, 533′, 534′, 535′ in this transformed captured image 550 are in thesame location as the border reference landmarks 532, 533, 534, 535 inthe reference pattern group 500 of FIG. 5A.

At step 440, the captured image is processed to determine an overalldistortion from a reference pattern to the pattern of the capturedimage. The overall distortion (i.e., the distortion introduced by thecorrective lens as well as the lens of the camera used to capture theimage) may be determined by comparing the patterns 510, 520 in thecaptured image 550 to the patterns in the reference pattern group 500.In some embodiments, the comparison is performed by comparing theplurality of pattern reference landmarks 512′ in the captured image 550to the plurality of pattern reference landmarks 512 in the referencepattern group 500.

FIG. 6 illustrates the locations of the plurality of pattern referencelandmarks 512′ in an exemplary captured image 550 overlaid over thelocations of the plurality of pattern reference landmarks 512 in thereference pattern group 500. The distance between each pattern referencelandmark 512 a′, 512 b′ in the captured image and its correspondingreference landmark 512 a, 512 b in the reference pattern group may beused to determine a dioptric power matrix P that describes thedistortion (i.e., transformation) from the ideal coordinate system tothe captured image.

Prentice's Rule describes the amount of induced prism in a lens. P canbe used to express Prentice's Rule in matrix form as x_(test)=Px_(ref),where x_(test) is a matrix of the location of the pattern referencelandmark 512 a′, 512 b′ in the captured image, and where x_(ref) is amatrix of the location of the corresponding reference landmark 512 a,512 b in the reference pattern group.

The dioptric power matrix P is given by:

$\begin{matrix}{{\text{?} = \begin{bmatrix}\text{?} & \text{?} \\\text{?} & \text{?}\end{bmatrix}}{\text{?}\text{indicates text missing or illegible when filed}}} & \lbrack 1\rbrack\end{matrix}$

where:

P _(x) =S+C sin² θ  [2]

P _(y) =S+C cos² θ  [3]

P _(z) =S+C sin θ cos θ  [4]

Solving algebraically allows for the determination of values for S, avalue related to the spherical power of the lens; C, a value related tothe cylinder power of the lens; and θ, the astigmatism angle of thelens.

The values of S and C describe the distortion introduced into thecaptured image by both the corrective lens and the lens of the camera ofthe lensmeter. Therefore, at step 450, the distortion of the pattern inthe captured image attributable to the corrective lens is determined. Inparticular, the focal lengths of the corrective lens along twoorthogonal axes corresponding to θ and θ+90°, f_(θ) and f_(θ+90°), aredetermined by the following equations when the corrective lens islocated at the halfway location between the camera and the pattern:

$\begin{matrix}{\mspace{79mu}{f_{\theta} = {\frac{l}{4}\left( \frac{\text{?}}{\text{?} - 1} \right)}}} & \lbrack 5\rbrack \\{\text{?} = {\frac{l}{4}\left( \frac{S + C}{S + C + 1} \right)}} & \lbrack 6\rbrack \\{\text{?}\text{indicates text missing or illegible when filed}} & \;\end{matrix}$

where l is the distance between the pattern and the lens of the cameraof the lensmeter. To determine the value of l, parameters of the cameraand/or lens may be determined or directly accessed from a data store. Insome embodiments, the focal length f of the camera lens may bedetermined from metadata in the captured image, or in configurationinformation for the lensmeter. The height h of the patterns may beknown. The distance l may be determined from f and other parameters.Methods and systems for determining a distance from an object (e.g., thepattern) are described in U.S. patent application Ser. No. 14/996,917,titled “SMARTPHONE RANGE FINDER” and filed on Jan. 15, 2016, the entiredisclosure of which is hereby incorporated by reference in its entirety.

At step 460, at least one characteristic of the corrective lens isdetermined. In some embodiments, the sphere, cylinder, and axismeasurements of the corrective lens may be determined, allowing for theprescription of the corrective lens to be determined. The spheremeasurement indicates the amount of lens power measured in diopters.Corrective lenses may be prescribed a certain sphere value to correctnearsightedness or farsightedness in all meridians of the eye. In someembodiment, the sphere value may be signed, with a negative valuerepresenting a nearsightedness prescription and a positive valuerepresenting a farsightedness prescription.

The cylinder value indicates the amount of lens power prescribed forastigmatism. The cylinder value may be zero if no correction isprescribed for astigmatism. A cylinder measurement indicates that thecorrective lenses have a first meridian with no added curvature, and asecond meridian, perpendicular to the first meridian, that contains amaximum lens curvature to correct astigmatism.

The axis value described the orientation of the second meridian of thecylinder. The axis value may range from 1° to 180°, with 90°corresponding to the vertical meridian of the eye, and 180°corresponding to the horizontal meridian.

Other values may also be determined for the corrective lenses, such asan ADD value representing an added magnifying power applied to thebottom part of a multifocal (e.g., bifocal or trifocal) lens.

In some embodiments, the sphere, cylinder, and axis measurements of thecorrective lens may be determined by the following equations:

$\begin{matrix}{\mspace{79mu}{{SPH} = {1000\text{/}f_{\theta}}}} & \lbrack 7\rbrack \\{\mspace{79mu}{{CYL} = {1000\mspace{11mu}\left( {\frac{1}{\text{?}} - \frac{1}{\text{?}}} \right)}}} & \lbrack 8\rbrack \\{\mspace{79mu}{{AXIS} = {\text{?} - \theta}}} & \lbrack 9\rbrack \\{\text{?}\text{indicates text missing or illegible when filed}} & \;\end{matrix}$

wherein the determination of AXIS is carried out from the perspective ofa wearer of the corrective lens.

The values of SPH, CYL, and AXIS may be displayed on a screen of thelensmeter, may be stored in a memory (e.g., a database, or a file) ofthe lensmeter, and/or may be delivered via the network interface of thelensmeter to another party, such as an eye doctor affiliated with anowner of the corrective lenses, for verification or for filling of theprescription. For example, the processes may be performed by a personwho has eyeglasses but does not know the prescription of those glasses.Information obtained through the methods discussed herein may betransmitted to the person's eyecare professional, who can use theinformation to order a new set of eyeglasses with the properprescription.

The process 400 ends at step 470.

In some embodiments, the requirement that the corrective lenses belocated halfway between the lensmeter and the pattern may be relaxed.The lensmeter and/or the corrective lenses may instead be moved relativeto each other and the pattern, with the lensmeter capturing multipleimages. For each image, values of S and C may be determined as discussedabove. It is known that S and S+C have an extreme value (i.e., a minimumor maximum) when the corrective lenses are positioned halfway betweenthe lensmeter and the pattern. The image for which S and S+C generate anextreme value may be used as the basis for the processes describedherein.

It will also be appreciated that, although the examples given hereinvolve corrective lenses in the form of eyeglasses, the processes andsystems may be applicable to other types of corrective lenses, such ascontact lenses, assuming the contact lenses can be held in a suitableorientation and location for performance of the claimed processes.

In some embodiments, the captured image is not transformed to an idealcoordinate system. Rather, two images are captured: a first image inwhich the corrective lens is disposed between the lensmeter and thepattern, as discussed in various embodiments herein, and a second“reference” image, identical to the first except that the correctivelens has been removed. Because the distortive effect of the lens is notpresent in the second image, the first image may be compared directly tothe second image to determine the amount of distortion using thetechniques discussed with respect to step 440 in process 400.

As discussed above (e.g., in connection with Equations 1-9), it can behelpful in fixtureless lensmeter operations for the corrective lens tobe placed halfway between the lensmeter camera and a displayed pattern.However, it can be challenging for a user to find the appropriatehalfway location.

As further noted above, the requirement that the corrective lens belocated halfway between the lensmeter and the pattern may be relaxed ifthe lensmeter and/or the corrective lenses are moved relative to eachother and the pattern, with the lensmeter capturing multiple imagesduring the movement for identification of extreme values for S and S+C,to identify one of the multiple images in which the corrective lens islocated at the halfway location.

However, it can be computationally expensive (e.g., in terms of CPUcycles, memory usage, and/or power) to capture, store, and/or processthe multiple images to identify the extrema of S and S+C.

In accordance with various aspects of the subject disclosure, improvedlensmeter systems and methods are provided in which the distance, l,between the pattern and the lens of the camera of the lensmeter and thedistance, l1, between the pattern and the corrective lens, and/or thedistance, l2 (e.g., l−l1) between the corrective lens and the patterncan be determined. Using one or more of these determined distances, theuser can be guided to place the corrective lens at the halfway location(e.g., l1=l2) before an analysis image is captured, and/or the lensmetercan compensate for the relative positions of the lensmeter, thecorrective lens, and the pattern.

The distances l, l1, and/or l2 can be determined by first determiningthe absolute size and shape of the corrective lens using a supplementalmeasurement image (sometimes referred to herein as a contact image or afirst image) captured with the corrective lens in contact with thepattern. FIGS. 7-11 illustrate various aspects of a fixturelesslensmeter operation using a contact image.

For example, FIG. 7 shows a perspective view of system 200 in whichlensmeter 210, two corrective lenses 220 mounted in a frame 708 of apair of eyeglasses 704, and pattern 230 are arranged for capture of acontact image. In the example of FIG. 7, pattern 230 is displayed in adisplay image 700 on a laptop computer 702. Laptop computer 702 may bepaired with lensmeter 210 (implemented in this example as a mobilephone). Lensmeter 210 may obtain display features for the display oflaptop 702 from laptop 702 and may provide instructions to laptop 702 tocontrol the display of image 700 and/or pattern 230.

For example, lensmeter 210 may obtain information associated with thedisplay of laptop 702 such as dimensional information indicating thesize (e.g., height and width) of the display of laptop 702 and/or thepixel size and density of the display. Using this information, lensmeter210 can determine and/or instruct laptop 702 to adjust the absolutephysical size of the features of pattern 230. In the example of FIG. 7,a single, screen-wide pattern 230 is displayed with embedded referencelandmarks 706. In this example, reference landmarks 706 may be utilizedas border reference landmarks (e.g., for orientation and transformationof captured images) as described above in connection with borderreference landmarks 532, 533, 534, and 535. In this example, patternreference landmarks such as pattern reference landmarks 512 can beprovided within pattern 230 and/or features of the pattern (e.g.,interfacing corners of pattern squares or other shapes) can be used aspattern reference landmarks.

In the operation illustrated in FIG. 7, while corrective lenses 220 arein contact with pattern 230 (e.g., in contact with the display of laptop702 while the display shows pattern 230), lensmeter 210 captures acontact image of corrective lenses 220, frame 708, and pattern 230through corrective lenses 220 and outside of frame 708. In theillustrated configuration, because corrective lenses 220 are in contactwith pattern 230, pattern 230 is substantially free of distortion bycorrective lenses 220. Accordingly, the sizes of the pattern features inthe contact image are known (e.g., and stored in memory 350) and thelocation of the edges of corrective lenses 220 (e.g., identified by theportions of pattern 230 that are blocked by frame 708) relative to thepattern 230 can be used to determine the absolute size and shape of eachcorrective lens.

As indicated in the example of FIG. 7, instructions 703 may be providedon a display of lensmeter 210. In other implementations, instructions703 may be provided entirely, or in part, on a display of laptop 702 oranother device. FIG. 8 illustrates an example of instructions 703 thatmay be provided during a lensmeter operation using lensmeter 210. Asshown in FIG. 8, instructions 703 may include a representation 800 ofborder reference landmarks 706, and a representation 802 of eyeglasses704 (including representations 804 and 806 of frame 708 and lenses 220)in position relative to representation 800 of border reference landmarks706, all based on one or more images captured by the camera of lensmeter210. As shown, text instructions 801 and/or graphical instructions 803may be included to instruct the user to place the eyeglasses 704 flatagainst pattern 230 (e.g., flat against the screen of laptop 702) as inthe configuration of FIG. 7, and to capture an image (e.g., snap aphoto).

As shown in FIG. 8, lens border indicators 810 may be displayed thatindicate the outer boundary of each of corrective lenses 220. Forexample, spectacles 704 may be segmented from the displayed pattern 230in an image captured by lensmeter 210, and the shapes corresponding tothe lenses 220 may be inferred on the basis of the location and relativesize of resulting connected components. Lens border indicators 810 canbe generated based on the inferred lens shape, and can then besuperimposed on a smartphone's camera view as indicated in FIG. 8 toprovide the user with feedback that the lens has been accuratelylocalized.

FIG. 8 also shows how lensmeter 210 may determine whether one or both oflenses 220 is partially occluded (e.g., by the user's finger or a templearm of the eyeglasses). For example, an occlusion may be detected bycomparing the lens shapes for two lenses 220 in a pair of eyeglasses 704using a shape mirroring and/or alignment procedure. An occlusion may bedetected when the mirrored shape of one lens is substantially different(e.g., smaller by more than a threshold) from the shape of the otherlens.

The precise localization and shape of the occlusion may be determined bylensmeter 210 by selecting points on the occluded lens shape that failto match the un-occluded lens shape. The occlusion boundary 814 definedby these selected points can then be superimposed on the smartphone'scamera view to inform the user of the nature and location of theocclusion so it can be addressed by the user. Additional instructions821 may be provided to instruct the user to remove the occlusion (if anocclusion is detected) or remind the user to avoid occluding the lenses.

As described above, when a contact image is captured with system 200 inthe configuration of FIG. 7 in which corrective lenses 220 are flatagainst the pattern 230, the boundaries (indicated in FIG. 8 by boundaryindicators 810) can be compared with the known sizes of pattern featuresin pattern 230 to determine the absolute size and shape of eachcorrective lens 220.

After capture of the contact image (e.g., response to user action, orautomatically as described above), instructions 703 may be changed toinstruct the user to move the corrective lenses 220 to a locationapproximately halfway between lensmeter 210 and pattern 230.

FIG. 9 illustrates a top perspective view of system 200 in aconfiguration in which the user has moved eyeglasses 704 such thatcorrective lenses 220 are located at an intermediate position 900between lensmeter 210 and pattern 230. FIG. 9 shows the distance, l1,between the pattern 230 and the corrective lenses 220, the distance, l2,between the corrective lenses 220 and the pattern 230, and the totaldistance, l, between the camera and the pattern.

Images captured during the motion and positioning of corrective lenses220 can be processed in real time to identify the shapes and sizes oflenses 220 and, by comparison with the shapes and sizes in the contactimage, determine the distances l1 and l2. One or more of thesedetermined distances and/or a graphical indicator of the absolute and/orrelative distances can be reported back to the user (e.g., using thedisplay of lensmeter 210) in real time, to help the user to guide thespectacles to a desired intermediate position 900 such as a halfwayposition at which the distance, l1, between the pattern 230 and thecorrective lenses 220, and the distance, l2, between the correctivelenses 220 and the pattern 230 are substantially the same (e.g., towithin a difference threshold).

Lens border indicators 810 and/or occlusion indicators 814 may also bedisplayed by lensmeter 210 in real time during the motion andpositioning of corrective lenses 220 to indicate to the user theposition of corrective lenses relative to reference landmarks 706 and toalert the user if one or more of the lenses is occluded.

When distances l1 and l2 are determined to be approximately the same(e.g., to within a difference threshold) based on the known sizes oflenses 220 from the processing of the contact image (e.g., and noocclusion is detected), lensmeter 210 may provide an indication that theuser has correctly positioned the lensmeter 210, the corrective lenses220, and the pattern 230, for example by displaying a “green light” orthumbs-up icon. When the desired positioning of corrective lenses 220has been achieved (e.g., at the halfway position), the user may beprovided with instructions to capture a lensmeter image (e.g., a lenspower measurement image) or the lensmeter image can be capturedautomatically. The characteristics of lenses 220 are measured (e.g., asdescribed above in connection with FIG. 3) using the captured lensmeterimage.

In this way, information obtained during positioning of the lensmeter210 and the lenses 220 can be reported back to the user in real-time(e.g., with reduced computation and power in comparison with repeatedlycomputing and comparing S and S+C as described above) to help the userinteractively adjust the arrangement of the spectacles and thesmartphone in order to achieve the desired imaging conditions. Thisinformation (e.g., the relative distances) can also be displayed for theuser after capture of a lensmeter image. The lensmeter can then providean option to retake or accept the lensmeter image, in someimplementations.

It should also be appreciated that, although FIGS. 7-9 depict a commonpattern across the entirety of the display of laptop 702, in variousscenarios, multiple patterns (e.g., two patterns such as patterns 510and 520 for two lenses of a pair of spectacles, or more than twopatterns for progressive or multi-focal lenses) can be used. However, asingle pattern as shown and described in connection with FIGS. 7-9 canalso be used (e.g., sub-region analysis of captured images) to determinevarious characteristics of multi-focal, progressive, or other complexlens arrangements. It should also be appreciated that, although a singlecontact image and a single lensmeter image are described in connectionwith FIGS. 7-9, in some circumstances, multiple contact images and/ormultiple lensmeter images can be captured and aggregated, averaged, orotherwise combined and/or co-processed to provide more accurate results.For example, multiple images can be stacked or otherwise combined foranalysis, and/or measurement results from multiple images can beaveraged or otherwise combined. For example, the lens power estimationoperations described herein are affected by the accuracy of measurementsof several quantities, such as the distortion of the test pattern incaptured images, and position and orientation of the pattern and lensesrelative to the lensmeter device. These measurements may be subject toinaccuracies which are, in part, uncorrelated from one moment to thenext. By acquiring a succession of similar images and aggregating thederived estimated measurements, the impact of the temporallyuncorrelated noise sources can be mitigated.

FIG. 10 depicts a flow diagram of an example process for lensmeteroperations, in accordance with various aspects of the subjecttechnology. For explanatory purposes, the example process of FIG. 10 isdescribed herein with reference to the components of FIGS. 2, 3 and 7-9.Further for explanatory purposes, some blocks of the example process ofFIG. 10 are described herein as occurring in series, or linearly.However, multiple blocks of the example process of FIG. 10 may occur inparallel. In addition, the blocks of the example process of FIG. 10 neednot be performed in the order shown and/or one or more of the blocks ofthe example process of FIG. 10 need not be performed.

In the depicted example flow diagram, at block 1000, a pattern such aspattern 230 is provided. As described above, the pattern may be aprinted pattern or may be provided by a computing device such as laptop702 (e.g., by the computing device itself and/or in cooperation with alensmeter device such a lensmeter device 210 implemented as a mobilephone).

At block 1002, instructions are provided (e.g., using a display of thelensmeter device) to place a corrective lens in contact (e.g., in flatcontact) with the pattern (see, e.g., FIGS. 7 and 9). Although theexample of FIG. 10 is described in connection with a single correctivelens, the operations of FIG. 10 can be performed for more than onecorrective lens, such as for a pair of corrective lenses mounted in aframe.

At block 1004, a first image (e.g., a contact image) of the pattern isobtained (e.g., by a camera of the lensmeter) through the correctivelens with the corrective lens in contact with the pattern. The firstimage may be obtained by capturing the image with the lensmeter deviceautomatically or responsive to a user's action as described herein.

At block 1006, the lensmeter identifies the outer boundaries of thecorrective lens using the first image (e.g., by segmenting thecorrective lens and/or a frame for the corrective lens from the patternin the first image and identifying the location of connected componentsin the segmented image to obtain points corresponding to the outerboundary). A boundary indicator such as boundary indicator 810 of FIG. 8may be displayed with a representation of the corrective lens asdescribed above.

At block 1008, the lensmeter determines whether the corrective lens isoccluded. Determining whether the corrective lens is occluded mayinclude identifying an irregular edge of a single corrective lens oridentifying a shape difference between mirrored shapes of two lenses ofa pair of eyeglasses (see, e.g., FIG. 8 and the associated discussion).

If it is determined that the corrective lens is occluded, the lensmetermay display, at block 1010, the first image with an occlusion indicatorsuch as occlusion indicator 814 of FIG. 8 associated with arepresentation 812 of the user's finger occluding the lens. Whileocclusion is detected, the lensmeter may continuously return to blocks1004, 1006, 1008, and 1010 until the occlusion is removed.

If it is determined that the corrective lens is not concluded, at block1012, the first image and the identified boundary may be stored (e.g.,in memory such as memory 350 of the lensmeter).

At block 1014, the size and shape of the corrective lens are determinedbased on the identified boundary and the pattern. For example, thepattern may be provided with features having a known shape and absolutesize so that the absolute dimensions of the corrective lens can bedetermined by comparison with the features of the pattern in the contactimage. For example, if the pattern includes a checkerboard pattern ofsquares, each having a width of X mm, and the largest dimension of thecorrective lens spans Y squares, then the largest dimension of thecorrective lens has a length of X*Y mm. One or more additionaldimensions of the lens can also be measured. Using this information, thelength of the largest dimension of the lens, in image pixels, in anysubsequent image can be used to determine the distance between thecamera and the corrective lens (e.g., if the pixel solid angle isknown). In combination with the pattern features of known size, alldistances l, l1, and l2 described above can be determined from thesecond image.

At block 1018, a second pattern that is different from the first patternmay be provided. The second pattern may include multiple patterns suchas patterns 510 and 520 for multiple lenses and/or one or moreadditional patterns for multi-focal or progressive lenses. However, ifdesired, the pattern displayed at block 1000 can remain displayed ratherthan displaying the second pattern.

At block 1020, the lensmeter may provide instructions to place thecorrective lens at an intermediate position 900, such as a halfwayposition, between the camera of the lensmeter device and the pattern.

At block 1022, a second image of the pattern (e.g., the first patternprovided at block 1000 or the second pattern provided at block 1018)through the corrective lens is obtained by the lensmeter with thecorrective lens at the intermediate position. The outer boundary of thecorrective lens may also be identified in the second image.

At block 1024, the lensmeter determines whether the corrective lens isoccluded.

If it is determined that the corrective lens is occluded, the lensmetermay display, at block 1026, the first image with an occlusion indicatorsuch as occlusion indicator 814 of FIG. 8 associated with arepresentation 812 of the user's finger occluding the lens. Whileocclusion is detected, the lensmeter may continuously return to blocks1022, 1024, and 1026 until the occlusion is removed.

If it is determined that the corrective lens is not occluded, at block1028, the distance l1, between the camera and the corrective lens andthe distance, l2 (e.g., l−l1), between the corrective lens and thepattern are determined (e.g., using the known size of the correctivelens as determined at block 1014, the second image, and/or the knownsize of the features of the pattern).

At block 1030, the lensmeter may determine whether the distance betweenthe lensmeter and the corrective lens and the distance between thecorrective lens and the pattern are similar to within a differencethreshold (e.g., whether distance l1 is approximately equal to distancel2). In this way, it may be determined whether the corrective lens is atthe desired intermediate position (e.g., the halfway position).

If it is determined that the corrective lens is not at the desiredintermediate position, at block 1032, reposition instructions may beprovided by the lensmeter. For example, the reposition instructions mayinclude a text-based and/or graphical display of the absolute and/orrelative distances l1 and l2 and/or an indicator such as a graphicalarrow or text instructing the user to move the corrective lens and/orthe lensmeter.

While the corrective lens is not at the intermediate position (e.g., asdetermined by the computation and comparison of the distances l1 andl2), the lensmeter may continuously repeat the operations of blocks1022, 1024, 1026, 1028, 1030, and/or 1032 until the corrective lens isat the desired intermediate location.

If it is determined, at block 1030, that the corrective lens is at thedesired intermediate position, the lensmeter may store, at block 1034,the second image of the pattern through the corrective lens. At block1030, the lensmeter may also provide an indication to the user that thecorrective lens is correctly positioned at the desired intermediatelocation by providing, for example a “green light” or thumbs-up icon.The second image may then be captured and stored automaticallyresponsive to the detection at the location or responsive to asubsequent image capture action by the user. In this way, the operationsof blocks 1030 and 1032 may help guide the user to place the correctivelens at the halfway location between the camera and the pattern, foroptimal measurements. However, it should also be appreciated that theoperation of blocks 1030 and 1032 may be omitted or truncated in somescenarios and the distance(s) determined at block 1028 can be used toaccount for differences in the distortion of the pattern caused by theposition of the corrective lens at different locations between thecamera and the pattern.

At block 1036, the second image may be transformed to an idealcoordinate system. For example, the second image may be transformed tothe ideal coordinate system represented by the reference pattern group706 of FIG. 7. This transformation may involve rotating, resizing,cropping, and skewing the second image to remove any distortions orimprecisions introduced by the image capture process. In someembodiments, the second image is transformed to an ideal coordinatesystem by detecting the border reference landmarks 706 in the secondimage, and transforming the second image using image manipulationtechniques to cause the border reference landmarks in the second imageto appear in the same location in the second image as the correspondingborder reference landmarks 706 in the reference pattern group. Theborder reference landmarks in the second image may be detected bycomputer vision techniques, and the detected border reference landmarksor the pixels constituting them may be configured to have a shape,color, or other characteristic suitable for performing such computervision techniques.

In some embodiments, a matrix transform is determined from the distancebetween the border reference landmarks in the second image and thecorresponding border reference landmarks 706 in the reference patterngroup. The matrix transform is then applied to some or all of the pixelsof the second image in order to effect the transformation

At block 1038, the second image is processed to determine an overalldistortion from a reference pattern to the pattern of the capturedimage. The overall distortion (i.e., the distortion introduced by thecorrective lens as well as the lens of the camera used to capture theimage) may be determined by comparing the pattern in the second image tothe pattern in the reference pattern. In some embodiments, thecomparison is performed by comparing a plurality of pattern referencelandmarks such as 512′ in the second image to a plurality ofcorresponding pattern reference landmarks such as pattern referencelandmarks 512 in the reference pattern group. The distortion of thepattern in the captured image attributable to the corrective lens maythen be determined (e.g., using the determined distance l, where l isdetermined using the second image and the known sizes of the features ofthe pattern and/or the determined size of the corrective lens). Thedistortion of the pattern in the captured image attributable to thecorrective lens may be determined using the determined distance l and,when the corrective lens is at the halfway location between the patternand the camera (e.g., when l1=l2=l/2), Equations 5 and 6 above. Incircumstances in which the correctively lens is not at the halfwaylocation (e.g., l1≠l2), the equations for determining the distortion ofthe pattern in the captured image attributable to the corrective lensare modified as would be understood by one skilled in the art, fordetermination of the distortion of the pattern in the captured imageattributable to the corrective lens.

At block 1040, at least one characteristic of the corrective lens isdetermined by the lensmeter. For example, the sphere, cylinder, and axismeasurements of the corrective lens may be determined, allowing for theprescription of the corrective lens to be determined. Other values mayalso be determined for the corrective lenses, such as an ADD valuerepresenting an added magnifying power applied to the bottom part of amultifocal (e.g., bifocal or trifocal) lens. In some embodiments, thesphere, cylinder, and axis measurements of the corrective lens may bedetermined by Equations 7, 8, and 9 above. Accordingly, the at least onecharacteristic of the corrective lens may be determined, at least inpart, based on the measured size of the corrective lens.

The values of SPH, CYL, and AXIS may be displayed on a screen of thelensmeter, may be stored in a memory (e.g., a database, or a file) ofthe lensmeter, and/or may be delivered via the network interface of thelensmeter to another party, such as an eye doctor affiliated with anowner of the corrective lenses, for verification or for filling of theprescription. For example, the processes may be performed by a personwho has eyeglasses but does not know the prescription of those glasses.Information obtained through the methods discussed herein may betransmitted to the person's eyecare professional, who can use theinformation to order a new set of eyeglasses with the properprescription.

Although various examples are described herein in which lensmeteroperations are performed by lensmeter device 210 (e.g., implemented as amobile phone or smart phone), it should be appreciated that some or allof the lensmeter operations may be performed remotely by a server usingimages and/or other information captured and transmitted by a mobiledevice. For example, FIG. 11 shows an implementation of a lensmetersystem that includes lensmeter 210 implemented as a mobile phone, alaptop computer 702 for displaying pattern 230, and a server 1130 incommunication with lensmeter 210 and/or laptop computer 702 via anetwork 1150.

As discussed above, aspects and functions disclosed herein may beimplemented as hardware or software on one or more of these computersystems. There are many examples of computer systems that are currentlyin use. These examples include, among others, network appliances,personal computers, workstations, mainframes, networked clients,servers, media servers, application servers, database servers and webservers. Other examples of computer systems may include mobile computingdevices, such as cellular phones and personal digital assistants, andnetwork equipment, such as load balancers, routers and switches.Further, aspects may be located on a single computer system or may bedistributed among a plurality of computer systems connected to one ormore communications networks.

For example, various aspects and functions may be distributed among oneor more computer systems configured to provide a service to one or moreclient computers. Additionally, aspects may be performed on aclient-server or multi-tier system that includes components distributedamong one or more server systems that perform various functions.Consequently, examples are not limited to executing on any particularsystem or group of systems. Further, aspects may be implemented insoftware, hardware or firmware, or any combination thereof. Thus,aspects may be implemented within processes, acts, systems, systemelements and components using a variety of hardware and softwareconfigurations, and examples are not limited to any particulardistributed architecture, network, or communication protocol.

Referring again to FIG. 3, the lensmeter 210 may be interconnected with,and may exchange data with, other systems such as server 1130 via thenetwork interface 340 connected to a network such as network 1150. Thenetwork may include any communication network through which computersystems may exchange data. To exchange data using the network, thelensmeter 210 and the network may use various methods, protocols andstandards, including, among others, Fibre Channel, Token Ring, Ethernet,Wireless Ethernet, Bluetooth, IP, IPV6, TCP/IP, UDP, DTN, HTTP, FTP,SNMP, SMS, MMS, SS7, JSON, SOAP, CORBA, REST and Web Services. To ensuredata transfer is secure, the lensmeter 210 may transmit data via thenetwork using a variety of security measures including, for example,TSL, SSL or VPN.

Various aspects and functions may be implemented as specialized hardwareor software executing in one or more computer systems. As illustrated inFIG. 3, the lensmeter 210 includes a camera 310, a processor 320, a userinterface 330, a network interface 340, a memory 350, and lensmetersoftware 360.

The processor 320 may perform a series of instructions that result inmanipulated data. The processor 320 may be a commercially availableprocessor such as an Intel Xeon, Itanium, Core, Celeron, Pentium, AMDOpteron, Sun UltraSPARC, IBM Power5+, or IBM mainframe chip, but may beany type of processor, multiprocessor or controller. The processor 320is connected to other system elements, including memory 350, the camera310, etc.

The memory 350 may be used for storing programs and data duringoperation of the lensmeter 210. Thus, the memory 350 may be a relativelyhigh performance, volatile, random access memory such as a dynamicrandom access memory (DRAM) or static memory (SRAM). However, the memory350 may include any device for storing data, such as a disk drive orother non-volatile storage device. Various examples may organize thememory 350 into particularized and, in some cases, unique structures toperform the functions disclosed herein.

The memory 350 may also include a computer readable and writeablenonvolatile (non-transitory) data storage medium in which instructionsare stored that define a program that may be executed by the processor320. The memory 350 also may include information that is recorded, on orin, the medium, and this information may be processed by the processor320 during execution of the program. More specifically, the informationmay be stored in one or more data structures specifically configured toconserve storage space or increase data exchange performance. Theinstructions may be persistently stored as encoded signals, and theinstructions may cause the processor 320 to perform any of the functionsdescribed herein. The medium may, for example, be optical disk, magneticdisk or flash memory, among others. A variety of components may managedata movement between the storage medium and other memory elements andexamples are not limited to particular data management components.Further, examples are not limited to a particular memory system or datastorage system.

The lensmeter 210 also includes one or more user interfaces 330. Theuser interface 330 may receive input or provide output. Moreparticularly, output devices may render information for externalpresentation. Input devices may accept information from externalsources. Examples of interface devices include keyboards, mouse devices,trackballs, microphones, touch screens, printing devices, displayscreens, speakers, network interface cards, etc.

Although the lensmeter 210 is shown by way of example as one type of acomputer device upon which various aspects and functions may bepracticed, aspects are not limited to being implemented on the lensmeter210 as shown in FIGS. 2 and 3. Various aspects and functions may bepracticed on one or more computers having a different architectures orcomponents than that shown in FIG. 3. For instance, the lensmeter 210may include specially programmed, special-purpose hardware, such as forexample, an application-specific integrated circuit (ASIC) tailored toperform a particular operation disclosed herein. While another examplemay perform the same function using a grid of several general-purposecomputing devices running MAC OS System X with Motorola PowerPCprocessors and several specialized computing devices running proprietaryhardware and operating systems.

The lensmeter 210 may include an operating system that manages at leasta portion of the hardware elements included in the lensmeter 210.Usually, a processor or controller, such as the processor 320, executesan operating system which may be, for example, a Windows-based operatingsystem, such as, Windows NT, Windows 2000 (Windows ME), Windows XP,Windows Vista or Windows 7 operating systems, available from theMicrosoft Corporation, a MAC OS System X operating system available fromApple Computer, one of many Linux-based operating system distributions,for example, the Enterprise Linux operating system available from RedHat Inc., a Solaris operating system available from Sun Microsystems, ora UNIX operating systems available from various sources. Many otheroperating systems may be used, and examples are not limited to anyparticular implementation.

The processor 320 and operating system together define a computerplatform for which application programs in high-level programminglanguages may be written. These component applications may beexecutable, intermediate, bytecode or interpreted code whichcommunicates over a communication network, for example, the Internet,using a communication protocol, for example, TCP/IP. Similarly, aspectsmay be implemented using an object-oriented programming language, suchas .Net, SmallTalk, Java, C++, Ada, or C# (C-Sharp). Otherobject-oriented programming languages may also be used. Alternatively,functional, scripting, or logical programming languages may be used.

Additionally, various aspects and functions may be implemented in anon-programmed environment, for example, documents created in HTML, XMLor other format that, when viewed in a window of a browser program,render aspects of a graphical-user interface or perform other functions.Further, various examples may be implemented as programmed ornon-programmed elements, or any combination thereof. For example, a webpage may be implemented using HTML while a data object called fromwithin the web page may be written in C++. Thus, the examples are notlimited to a specific programming language and any suitable programminglanguage could be used. Thus, functional components disclosed herein mayinclude a wide variety of elements, e.g. executable code, datastructures or objects, configured to perform described functions.

Embodiments described above utilize a process for determiningcharacteristics of a corrective lens using a camera of a mobile device.Other embodiments may be used to determine characteristics of a lens ina number of different applications including: detecting flaws in a lens;comparing characteristics of two different lenses; determining thestructural characteristics of the lens based on the amount ofdiffraction (i.e., distortion) detected; or other applications that callfor determining the characteristics of a lens.

Having thus described several aspects of at least one example, it is tobe appreciated that various alterations, modifications, and improvementswill readily occur to those skilled in the art. For instance, examplesdisclosed herein may also be used in other contexts. Such alterations,modifications, and improvements are intended to be part of thisdisclosure, and are intended to be within the scope of the examplesdiscussed herein. Accordingly, the foregoing description and drawingsare by way of example only.

1.-20. (canceled)
 21. A method of operating a lensmeter system,comprising: capturing, with a camera of the lensmeter system, a firstimage of a pattern through a corrective lens that is in contact with thepattern; determining, with a computing device of the lensmeter system, asize and a shape of the corrective lens based on the first image and thepattern; capturing, with the camera, a second image of the patternthrough the corrective lens while the corrective lens is at anintermediate position between the camera and the pattern; determining,with the computing device, border reference landmarks of the pattern;detecting, with the computing device, border reference landmarks of thesecond image; transforming, with the computing device, the second imageof the pattern based on the border reference landmarks; determining,with the computing device, a distortion, attributable to the correctivelens, of the pattern in the second image, using the determined size andshape of the corrective lens and based on the transformation of thesecond image; and measuring, with the computing device, at least onecharacteristic of the corrective lens based on the determineddistortion.
 22. The method of claim 21, wherein the pattern includesfeatures having a known size, and wherein determining the size of thecorrective lens comprises: identifying an outer boundary of thecorrective lens in the first image; and determining a number of thefeatures having the known size that are located within the outerboundary in the first image, wherein the features having the known sizethat are located within the outer boundary in the first image are freeof distortion by the corrective lens due to the contact of thecorrective lens with the pattern.
 23. The method of claim 21, furthercomprising, while the corrective lens is at the intermediate position:determining with the computing device, a first distance between thecorrective lens and the camera using the determined size and the secondimage; and determining, with the computing device, a second distancebetween the corrective lens and the pattern using the determined sizeand the second image.
 24. The method of claim 23, wherein determiningthe distortion using the determined size comprises determining thedistortion using at least one of the first distance and the seconddistance.
 25. The method of claim 23, further comprising providinginstructions, with a display of the lensmeter system and based on thefirst distance and the second distance, to move the camera or thecorrective lens.
 26. The method of claim 25, wherein the instructionsadapt to movement of the camera or the corrective lens to guide a userof the lensmeter system to change the intermediate position to a halfwayposition between the camera and the pattern.
 27. The method of claim 23,further comprising: providing, with a display of the lensmeter system,at least a representation of the first distance and the second distance;and providing an option to retake the second image.
 28. The method ofclaim 21, further comprising determining, with the computing device,whether the corrective lens is occluded in the second image.
 29. Themethod of claim 28, further comprising displaying, based on the secondimage, a representation of the corrective lens and an outer boundary ofthe corrective lens, and an occlusion indicator that indicates anocclusion of the corrective lens.
 30. The method of claim 29, whereinthe corrective lens comprises one of two corrective lenses in a pair ofspectacles in the second image, and wherein determining whether thecorrective lens is occluded comprises determining whether the outerboundary of the corrective lens has a shape that is a mirror of a shapeof the other of the two corrective lenses.
 31. The method of claim 21,wherein capturing the first image comprises capturing a plurality offirst images while the corrective lens is in contact with the pattern,and wherein determining the size of the corrective lens comprisesaggregating information associated with the plurality of first images.32. The method of claim 21, wherein the camera comprises a camera of thecomputing device, wherein the computing device comprises a mobile phone,and wherein the method further comprises displaying the pattern with adisplay of a different computing device.
 33. The method of claim 32,further comprising providing information associated with the display ofthe different computing device from the different computing device tothe mobile phone.
 34. A mobile device, comprising: a camera; and aprocessor configured to: obtain, from the camera, a first image of apattern through a corrective lens that is in contact with the pattern;determine a size and shape of the corrective lens based on the firstimage and the pattern; obtain, from the camera, a second image of thepattern through the corrective lens with the corrective lens at anintermediate position between the camera and the pattern; determineborder reference landmarks of the pattern; detect border referencelandmarks of the second image; transform the second image of the patternbased on the border reference landmarks; determine a distortion,attributable to the corrective lens, of the pattern in the second image,using the determined size and shape of the corrective lens and based onthe transformation of the second image; and measure at least onecharacteristic of the corrective lens based on the determineddistortion.
 35. The mobile device of claim 34, further comprising anetwork interface, wherein the processor is further configured tocommunicate with a remote computing device via the networking interfaceto coordinate display of the pattern by the remote computing device. 36.The mobile device of claim 34, further comprising a display, wherein theprocessor is further configured to operate the display to: provideinstructions to place the corrective lens in contact with the patternand to capture the first image; and provide instructions to move thecorrective lens to the intermediate position and to capture the secondimage.
 37. A lensmeter system, comprising: a pattern having featureswith sizes; and mobile device, comprising: a camera; memory storinginformation associated with the pattern and the sizes of the features;and a processor configured to: capture, using the camera, a first imageof a pattern through a corrective lens that is in contact with thepattern; determine a size and shape of the corrective lens based on thefirst image and the pattern using the information associated with thepattern and the sizes and shapes of the features; capture, using thecamera, a second image of the pattern through the corrective lens whilethe corrective lens is at an intermediate position between the cameraand the pattern; determine border reference landmarks of the pattern;detect border reference landmarks of the second image; transform thesecond image of the pattern based on the border reference landmarks;determine a distortion, attributable to the corrective lens, of thepattern in the second image, using the determined size and shape of thecorrective lens and based on the transformation of the second image; andmeasure at least one characteristic of the corrective lens based on thedetermined distortion.
 38. The lensmeter system of claim 37, wherein thepattern comprises a printed pattern.
 39. The lensmeter system of claim37, further comprising a computer having a display, wherein the patterncomprises a pattern displayed by the display of the computer.
 40. Thelensmeter system of claim 39, wherein the mobile device is configured toreceive dimensional information about the display from the computer andto determine the size of the corrective lens using the dimensionalinformation, the first image, and the known sizes of the features of thepattern.